---
title: Re-Ranking Evaluations
---
Re-ranking involves reordering nodes based on relevance to the query and choosing top n nodes. Different evaluations are performed based on the number of nodes returned after re-ranking.
  1. Same Number of Nodes

      Context Reranking: Checks if the order of re-ranked nodes is more relevant to the query than the original order.
  2. Different Number of Nodes:

      Context Conciseness: Examines whether the reduced number of nodes still provides all the required information.
  
These evaluations collectively ensure the robustness and effectiveness of the RAG query engine, SubQuestionQueryGeneration operator, and the re-ranking process in the LlamaIndex pipeline.

### How to do it?
<Steps>
  <Step title="Install UpTrain and LlamaIndex">
```python
pip install -q html2text llama-index pandas tqdm uptrain cohere
```
  </Step>
  <Step title="Import required libraries">
```python
from llama_index import (
    ServiceContext,
    VectorStoreIndex,
)
from llama_index.node_parser import SentenceSplitter
from llama_index.readers import SimpleWebPageReader
from llama_index.callbacks import CallbackManager, UpTrainCallbackHandler
from llama_index.postprocessor.cohere_rerank import CohereRerank
from llama_index.service_context import set_global_service_context
from llama_index.query_engine.sub_question_query_engine import (
    SubQuestionQueryEngine,
)
from llama_index.tools.query_engine import QueryEngineTool
from llama_index.tools.types import ToolMetadata
```
  </Step>
  <Step title="Setup UpTrain Open-Source Software (OSS)">
  You can use the open-source evaluation service to evaluate your model. In this case, you will need to provie an OpenAI API key. You can get yours [here](https://platform.openai.com/account/api-keys).

Parameters:
* `key_type`="openai"
* `api_key`="OPENAI_API_KEY"
* `project_name_prefix`="PROJECT_NAME_PREFIX"
```python
callback_handler = UpTrainCallbackHandler(
    key_type="openai",
    api_key="sk-...",  # Replace with your OpenAI API key
    project_name_prefix="llama",
)
Settings.callback_manager = CallbackManager([callback_handler])
```
  </Step>
  <Step title="Load and Parse Documents">
  Load documents from Paul Graham's essay "What I Worked On".
  ```python
  documents = SimpleWebPageReader().load_data(
    [
        "https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/paul_graham/paul_graham_essay.txt"
    ]
)
  ```
  Parse the document into nodes.
```python
parser = SentenceSplitter()
nodes = parser.get_nodes_from_documents(documents)
```
</Step>

 <Step stepNumber = '5' title = 'Re-ranking' >
  Re-ranking is the process of reordering the nodes based on their relevance to the query. There are multiple classes of re-ranking algorithms offered by Llamaindex. We have used CohereRerank for this example.

  The re-ranker allows you to enter the number of top n nodes that will be returned after re-ranking. If this value remains the same as the original number of nodes, the re-ranker will only re-rank the nodes and not change the number of nodes. Otherwise, it will re-rank the nodes and return the top n nodes.

  We will perform different evaluations based on the number of nodes returned after re-ranking.



  <Step stepNumber = '5.1'>
  <p> <strong> Re-ranking (With same number of nodes) </strong></p>
If the number of nodes returned after re-ranking is the same as the original number of nodes, the following evaluation will be performed:

  * Context Reranking: Check if the order of the re-ranked nodes is more relevant to the query than the original order.
```python
api_key = "**********************************"  # Insert cohere API key here
cohere_rerank = CohereRerank(
    api_key=api_key, top_n=5
)  # In this example, the number of nodes before re-ranking is 5 and after re-ranking is also 5.

index = VectorStoreIndex.from_documents(
    documents=documents, service_context=service_context
)

query_engine = index.as_query_engine(
    similarity_top_k=10,
    node_postprocessors=[cohere_rerank],
    service_context=service_context,
)

response = query_engine.query(
    "What did Sam Altman do in this essay?",
)
```
```bash
Question: What did Sam Altman do in this essay?
Context Reranking Score: 0.0
```
</Step>
  <Step stepNumber = '5.2'>
<p> <strong> Re-ranking (With different number of nodes) </strong></p>
  If the number of nodes returned after re-ranking is the lesser as the original number of nodes, the following evaluation will be performed:

  * Context Conciseness: If the re-ranked nodes are able to provide all the information required by the query.
```python
api_key = "**********************************"  # insert cohere API key here
cohere_rerank = CohereRerank(
    api_key=api_key, top_n=2
)  # In this example, the number of nodes before re-ranking is 5 and after re-ranking is 2.

index = VectorStoreIndex.from_documents(
    documents=documents, service_context=service_context
)
query_engine = index.as_query_engine(
    similarity_top_k=10,
    node_postprocessors=[cohere_rerank],
    service_context=service_context,
)

# Use your advanced RAG
response = query_engine.query(
    "What did Sam Altman do in this essay?",
)
```
```bash
Question: What did Sam Altman do in this essay?
Context Conciseness Score: 1.0
```
  </Step>
  </Step>
</Steps>


<CardGroup cols={2}>
  <Card
    title="Tutorial"
    href="https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/docs/examples/callbacks/UpTrainCallback.ipynb"
    icon="infinity"
    color="#808080"
  >
    Open this tutorial in Colab
  </Card>
  <Card
    title="Have Questions?"
    href="https://join.slack.com/t/uptraincommunity/shared_invite/zt-1yih3aojn-CEoR_gAh6PDSknhFmuaJeg"
    icon="slack"
    color="#808080"
  >
    Join our community for any questions or requests
  </Card>
</CardGroup>
